Approachability, fast and slow

Approachability has become a central tool in the analysis of repeated games and online learning. A player plays a repeated vector-valued game against Nature and her objective is to have her long-term average reward inside some target set. The celebrated results of Blackwell provide a 1/ √ n convergence rate of the expected point-to-set distance if this is achievable, i.e., if the set is approachable. In this paper we provide a characterization for the convergence rates of approachability and show that in some cases a set can be approached with a 1/n rate. Our characterization is solely based on a combination of geometric properties of the set with properties of the repeated game, and not on additional restrictive assumptions on Nature’s behavior. Introduction Approachability goes back to the seminal paper of Blackwell (1956a) who considered a repeated game between a player and Nature. The stage outcome is a vector-valued reward and Blackwell provided sufficient conditions – that happened to be necessary in some cases – under which the player can guarantee that, asymptotically, her average reward vector belongs to some fixed (convex) target set. Such a set is then called approachable, and Blackwell also exhibited a strategy ensuring that the rate of convergence is independent of the dimension of the space (although it depends on the maximal norm of possible rewards) as, no matter the sequence of moves of Nature, the distance at stage n of the average payoff to an approachable set is smaller than O(n−1/2). Approachability theory is now a standard tool in the analysis of repeated games. For example, Kohlberg (1975) proved that it can be used to construct optimal strategies in a class of games with incomplete information; see also Aumann and Maschler (1995); Mertens et al. (1994). It is also widely studied in machine learning, as Blackwell (1956b) himself noticed that regret minimization, introduced by Hannan (1957) earlier, can be easily described as a special instance of approachability. The number of modifications, generalizations and improvements of his original arguments has increased dramatically during the last years; see Hart and Mas-Colell (2001); Cesa-Bianchi and Lugosi (2006); Abernethy et al. (2011); Perchet (2013) and references therein. One of the advantages of approachability theory is that it allows to treat and solve not only usual regret minimization, but also with extra assumptions (cost constraints, variable stage duration, etc.; see Mannor and Shimkin (2008); Mannor et al. (2009); Perchet (2013)) as well as other online learning problems such as ∗ We thank Jacob Abernethy for asking the original question during COLT ’11. Vianney Perchet acknowledges funding from the ANR, under grant ANR-10-BLAN-0112. This research was partially supported by the Israel Science Foundation under grant no. 920/12. c © 2013 S. Mannor & V. Perchet.